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EMBEDDING WATERMARKS FOR PROTECTING MULTIPLE COPIES OF A SIGNAL 



The present invention generally relates to the field of watermarking of media 
signals and more particularly to methods, devices and a media signal for simplifying 
embedding of watermarks in different copies of a media signal. 



It is widely known to watermark media signals in order to prevent illegal 
copying and distribution of the media signals. In this way a rightful owner of a media signal 
can detect if for instance copies of a media signal have been illegally made. 

With the introduction of Internet, there has been revolution within the filed of 

10 distribution of media content, through allowing downloading of media signals from ' 
Electronic Content Delivery Systems. A content provider can then have a database of 
difiTerent media signals and deliver copies of these to difiTerent users via the Internet A 
purchaser of this content then often expects an instantaneous or direct delivery of access to . 
the content. In order to safeguard illegal copying of these media signals it is then necessary to 

1 5 watermark each copy distributed with a unique watermark. 

Watermarking is however in many cases relatively time consuming and 
involving complex computing operations. This means that if a content provider Is to deliver 
many copies of a media signal at the same time to different clients, the watermarking process 
for each copy of the media signal will take a long time, which can lead to many clients 

20 receiving their ordered media signal after a considerable delay, which is in many cases not 
acceptable. 

There is therefore a need for providing watermarking of different copies of a 
media signal that is faster, while still guaranteeing unique watermarks for each watermarked 
copy. 

25 The document "Real-time Concepts for Block-based Watermarking Schemes" 

by Michael Arnold and Oliver Lobisch, WEDELMUSIC Conference, 2002, Darmstadt, 
Germany, pp. 156 - 160, describes one system for reducing the computational time used. 
Here a media signal is watermarked with two different watermarks. The two watermarked 
copies of the media signal are then stored. Just prior to distributing the signal to different 
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clients^ the watermarked signals are then mixed in a client dependent way in order to provide 
unique watermarks for different clients. This technique works best when the media signal is 
provided in frames. Then at every frame boundaiy one out of the two watermarked signals 
are mixed into the output signal. In this way an output media signal can be provided such that 
5 a sequence of the two watermarics are provided on a frame-per-frame basis. The specific 
sequence then represents the watermaric of a certain client. The document furthermore 
describes the use of a secret key that determines the order in which the two watermarks are 
provided. A similar system is also described in the document WO00/S60S9. 

Many types of watermaridng do not use frames. In for instance mask coding 

10 there is a continuous sequence of symbols. Mixing of watermarics is possible in this 

environment The mixing is however difficult to perform and such mixed watermarks would 
furthermore be difficult to detect. The payload capacity is furthermore limited by tfie 
maximum number of possible changes in the watermarking sequence. If the watermarking 
algorithm properties do not allow a relatively fast change in watermark and/or content size is 

1 5 relatively small, the total payload may be limited. The storage capacity for storing the 
watermarks furthermore needs to be increased significantly because of the storing space 
requirements for the'two watermarks. 

There is thus a need for an alternative way of providing watermarking of 
different copies of a media signal that is faster, while still guaranteeing unique watermarks 

20 for each watermarked copy and that furthermore limits the storage capacity needed, allows a 
bigger variation of unique watermarks, especially for small sized media content, and that can 
be used for also other watermarking techniques that do not use frames. 



25 It is thus an object of the present invention to provide watermarking of 

different copies of a media signal that is fast, while still guaranteeing unique watermarks for 
each watermarked copy, limits the storage capacity needed, allows a bigger variation of 
unique watermarks and that can be used for watermarking techniques that do not use frames. 

According to a first aspect of the present invention, this object is achieved by a 
30 method of simplifying embedding of watermarks in different copies of a media signal 
comprising the steps of: 

determining watermarking properties dependent on a media signal, and 
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Storing the signal dependent properties, such that the signal dependent 
properties can be used when embedding unique watermarks in different copies of the media 
signal. 

According to a second aspect of the present invention, this object is also 
S achieved by a method of embedding a watermark in a media signal comprising the steps of: 

receiving a media signal together with certain watermarking properties 
dependent on the media signal, and 

embedding a watermark based on the signal dependent properties in a copy of 
the media signal. 

10 According to a third aspect of the present invention, this object is furthermore 

achieved by a device for simplifying the embedding of watermarks in different copies of a 
media signal comprising a server unit including: 

a properties determining unit for determining signal dependent watermarking 
properties of a media signal , and 
1 5 - a signal properties store for storing the signal dependent properties, such that 

the signal dependent properties can be used for embedding unique watermarks in different 
copies of the media signal. 

According to a fourth aspect of the present invention, this object is 
furthermore achieved by a device for embedding a watermark in a media signal comprising: 
20 - a receiving unit for receiving a media signal together with certain 

watermarking properties dependent on the media signal, and 

a watermarking unit arranged to embed a watermark based on the signal 
properties in a copy of the media signal. 

According to a fifth aspect of the present invention, this object is also achieved 
25 by a signal for providing media content to a recipient comprising a media signal together 
with certain watermarking properties, which are dependent on the media signal. 

Claims 2 and 12 are directed towards sending the media signal together with 
information that is at least based on the signal dependent properties to a recipient. 

Claims 3 and 13 are directed towards embedding watermarks in copies of the 
30 media signal using the signal dependent properties and then sending the media signal 

including a unique watermark to recipients. In this way watermarks are embedded before 
delivery to recipients. 

Claims 4 and 14 are directed towards mixing watermarks in copies of the 
media signal. This enables the provision of unique watermarks in several copies of a media 
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signal based on a limited number of original watermarlcs with only limited additional 
processing. 

Claims S and IS are directed towards sending die media signal and the signal 
dependent properties to a recipient In this way a trusted client can himself embed a unique 
5 watermark in the media signal. This also allows inspection of effects on the watermarking on 
the media signal beforehand, which allows modification of the signal dependent properties of 
a watermaric before embedding. 

Claims 6, 10, 16 and 19 are directed towards lossless encoding and decoding 
of the signal dependent properties in the media signal. This has the advantage of not requiring 
1 0 any extra bandwidth for the transmission of the signal dependent properties. 

Claims 7 and 17 are directed towards using a perceptual model of a human 
sensing system for determining the signal dependent properties. 

The present invention has the advantage of allowing embedding of watennarks 
in multiple copies of the same media signal with small time delays. The storage requirements 
15 are furthermore smaller than if several parallel watermaiics are stored. 

The general idea behind the invention is thus to split the watermarking process 
into two parts, one which is based on signal dependent properties and one which is based on 
watermark specific properties. TTie signal dependent properties are determined in advance, 
while Ae watermark specific properties are determined at the time of delivery of the media 
20 signal. 

These and other aspects of the invention will be apparent from, and elucidated 
with reference to, the embodiments described hereinafter. 

25 The present invention will now be explained in more detail in relation to the 

enclosed drawings, by way of example, where 

Fig. 1 shows a block schematic of a device for simplifying embedding of 
watermarks in copies of a media signal according to a first embodiment of the invention. 
Fig. 2 shows a flow chart of the method of simplifying embedding of 
30 watermarks in a media signal the device in Fig. 1 is working according to. 

Fig. 3 shows a block schematic of a watermarking unit that can be used in the 
device in Fig. 1, 

Fig. 4 shows a block schematic of a device for simplifying embedding of 
watermarks in copies of a media signal according to a second embodiment of the invention. 
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Fig. 5 shows a block schematic of a server device for simplifying embedding 
of watermarlcs in copies of a media signal together with a corresponding client device 
according to a third embodiment of the invention. 

Fig* 6 shows a flow chart of a method of simplifying embedding of 
watermarlcs in a media signal, that the server device in Fig. S is working according to. 

Fig. 7 shows a flow chart of a method of simplifying embedding of 
watermarks in a media signal, that the client device in Fig. 5 is working according to, and 

Fig. 8 shows a block schematic of a server device for simplifying embedding 
of watermarks in copies of a media signal together with a corresponding client device 
according to a fourth embodiment of the invention. 



The present invention relates to the field of providing watermarks in media 
signals and is especially arranged for provision of watermarks in electronic media signal 
delivery systems. 

Fig. 1 shows a block schematic of a device for simplifying embedding of 
watermarks in copies of a media signal according to a first embodiment of the invention. The 
device includes a server unit 10 and a media signal delivery unit 20, which are connected to 
each other. The functioning of the device will now also be described in relation to Fig. 2, 
which shows a flow chart of the method the device is working according to. The server unit 
includes a media store MS 12 including a number of media signals, such as for instance a 
number of audio signals, like a number of songs. The media store 12 is connected to an 
analyzing unit 14, which determines watermarking properties p that are based on the media 
signal X to be watermarked, step 30. After having determined these properties the analyzing 
unit 14 stores these properties p in a property store PS 16, step 32. This determination is 
performed for all signals in the store 12 and is furthermore performed off-line, i.e. in advance 
of any delivery or request for delivery of a media signal to a client or recipient. 

When the device receives requests for certain media content, the server unit 
forwards the media signal x from the signal store and the properties p corresponding to the 
signal to a number of watermarking units 22, 24, 26 in the signal delivery unit 20, step 34. In 
Fig. 1, there are three such units shown, each embedding a unique watermark Wa, Wb, Wc in 
the media signal x, step 36. The watermarks Wa, Wb, Wc can here be seen as watermark 
dependent properties. When performing this embedding the watermarking units use the signal 
dependent properties p. How this can be done will shortly be described in more detail. 
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Thereafter the watermarked signals Xa, Xb» Xc are provided to a sending unit 28, which sends 
the thus watermarked media signals Xa» Xb> Xc to the recipients, step 38. This sending is in 
Fig. 1 indicated with a thick arrow. The actual watermarking is here provided on-line, i.e. 
when the signal is to be delivered, so that watermarking is taking place just before a signal is 

S to be delivered, which enables delivery on demand even if many clients wish to receive a 
media signal. The sending is of course also provided on-line. The sending unit is a normal 
interface adapted to be used for the transmission medium in question, which in the present 
example is the Internet. 

With this device a number of advantages are obtained. It is possible to embed 

10 watermarks in multiple copies of the same media signal with small time delays, because the 
signal dependent properties have been determined beforehand. The storage requirements are 
furthermore smaller than if several parallel watermarks are stored, since only one copy of 
these signal properties are stored, which are then used for all watermarks embedded ui a copy 
of the media signal. 

IS There are some variations of this first embodiment that are obvious. First of all 

it is possible to have only one watermarking unit instead of three, which could provide all the 
watermarking. Then it would be having different watermarks as input information. It should 
therefore also be realized that more or fewer watermarking units can be used in parallel. 
Another obvious modification is that the units 10 and 20 can be combined into one single 

20 unit They can also be provided in a network, which would then preferably be a separate 
network from the network on which the signal is sent to the client, although this is no 
requirement. It is also possible to provide several units 20, each communicating with the 
server 10. 

How an actual watermarking unit can be made to work will now be described 
25 with reference being made to Fig. 3, which shows a block schematic of an envelope 

modulation watermarking unit 22. The device is a device for watermarking of signal samples, 
like for instance of audio signal samples like PCM samples. This is however just one 
example of the type of signals in which watermarking according to the invention can be 
performed. It should be realized that the watermarking principles of the present invention can 
30 just as well be applied on compressed audio signals as well as on image or video signals, 

either compressed or decompressed. The watermarking unit 22 includes a bandpass filter 44, 
which filters the media signal x[n] and provides the filtered signal Xb[n] to a multiplying unit 
42, which also receives a watermark Wa[n] and multiplies the watermark Wa[n] with the 
filtered media signal Xb[n]. The output of the multiplying unit 42 is connected to a scaling 
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unit 46, which scales the output signal from the multiplying unit 42 with a scaling parameter 
a and provides it to an adding unit 48, which also receives the media signal x[n]. The output 
of the adding unit 48 is then the watermarked media signal XA[n]. The scaling factor a is 
controlled by a signal p[n], which signal is made up of the watermarking properties 

5 depending on the media signal. These properties are in this example decided based on a 
psycho-acoustic model of the human hearing system in order to ensure that the watermark is 
not perceptible to a user or client or provided beneath a masking threshold of the signal x[n]. 
These properties, which are highly dependent on the media signal x, are therefore calculated 
beforehand and stored in the properties store in Fig. 1. The calculation of these properties is 

10 quite complex and time demanding, but is however only needed to be made once for each 
signal. Also the bandpass filtered signal Xb[n] is fixed m the sense that it is not influenced by 
the watermark and can just as well be calculated beforehand and stored in the properties 
store. This will make the watermarking unit simpler in structure but also require additional 
storage space in the server. It should be realized that the watermarking properties dependent 

15 on the media signal are not limited to being based on a psycho-acoustic model of the human 
hearing system. In case the media signal is a still image or a video signal an appropriate 
psycho-visual model of the human visual system is used. The model is therefore a model of a 
human sensing system. More detail about the specific watermarking technique shown in 
Fig. 3 can be found in the document, "A temporal domain audio watermarking technique", by 

20 Aweke Negash Lemma, Javier Aprea, Werner Oomen and Leon van de Kerkhof, IEEE 
Transactions on Signal Processing, April 2003, Vol. 51, page 1088-1097, which is herein 
incorporated by reference. 

According to a second embodiment of the invention, it is possible to also mix 
together two watermarks in order to increase the payload capacity of the watermark. A device 

25 for simplifying embedding of watermarks in copies of a media signal according to this 

second embodiment of the invention is shown in a block schematic in Fig. 4. Here the server 
unit 10 is identical with the server unit in Fig. 1 . The notable difference here is that the signal 
delivery unit 20 includes two watermarking units 22 and 24 instead of three as well as a 
mixing unit 50 provided between the sending unit 28 and the watermarking units 22, 24. The 

30 server unit 1 0 here works in the same way as the server unit in Fig. 1 and will therefore not 
be described in more detail. Also the watermarking units 22 and 24 work in the same way as 
has been described in relation to Fig. 1, and will therefore not be described in more detail. 
The mixer 50 receives the two watermarked copies Xa and Xb of the signal x and mixes them 
so that different sections of the signal x is watermarked with one of the watermarks in a 
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sequence determined by the mixer. In Fig. 4 only four sections are indicated in the output 
signals from the mixer. It should however be understood that more or fewer sections could be 
provided. The mixer thus outputs different combinations of the watermarks Wa and Wb, 
embedded in copies of the media signal, where the number of combinations in the given 
S example can be sixteen. In the figure only three are however shown xa,b>v,b, Xama* xb>^^3* 
It should also be understood that if more sections are used the number of possible 
combinations are increased. Another possible variation is to mix more than two watermarked 
signals. The thus mixed signals are then provided to the sending unit 28 for sending to 
different clients. How this mixing can be performed in more detail is described in the 
10 document **Real-time Concepts for Block-based Watermarking Schemes** by Michael Arnold 
and Oliver Lobisch, WEDELMUSIC Conference, 2002, Darmstadt, Germany, pp. 156 - 160, 
which is herein incorporated by reference. 

With the device in Fig. 4, the payload capacity is increased. Hie device 
furthermore allows the provision of unique watermarks in several copies of a media signal 
IS based on a limited number of original watermarks with only limited additional processing 
and thus a slightly slower watermarking speed. 

The first and second embodiment were described in relation to a system of 
delivering media signals to clients, where the system embeds watermarks before supply to a 
client. This system is a consumer system, where the client is not a trusted party. The present 
20 invention is however also applicable in systems where the client is a trusted type of client, 
which can for instance be a company. This system can for example be used for delivering 
video signals carrying movie information to a cinema. 

A third embodiment of the present invention is directed towards this type of 
environment. A server for simplifying embedding of watermarks in copies of a media signal 
25 together with a client device performing the watermark embedding according to this third 
embodiment of the invention is shown in a block schematic in Fig. 5. The functioning of this 
system will now also be described in relation also to Fig. 6 and 7, which show flow charts of 
the method the server and the client devices are working according to. 

As mentioned above the server unit includes a media store MS 12 including a 
30 number of media signals. The media store 12 is connected to the analyzing unit 14, which 
determines watermarking properties p that are based on the media signal x to be 
watermarked, step 58. After having determined these properties the analyzing unit 14 stores 
tfiese properties p in the property store PS 16, step 60. This determination is performed for all 
signals in the store 12 and is furthermore performed off-line, i.e. in advance of any delivery 
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of the media signal to the client. The server unit 10 also includes a sending unit 52 which 
sends the media signal x as well as the watermark properties p that depend on the media 
signal, step 62. This sending is in Fig. S indicated with a thick arrow. The information is 
furthermore preferably sent using encryption. 

The client device 54 includes a receiving unit 56, where the properties and the 
media signal are received, step 64. The client device also includes a watermarking unit 22. 
The receiving unit 56 therefore provides the media signal x and the properties p to this 
watermarking unit, step 66. The watermarking unit thereafter embeds a unique watermaric 
using the signal dependent properties p in the media signal, step 68. The media signal can 
then be used for display to an audience in a movie theatre if it is a video signal. Different 
clients can then embed their own unique watermark in the media signals they receive. The 
watermarking unit can furthermore be arranged to embed a watermark that is dependent on 
the date and time, such that the watermark of a copy of the media signal that is displayed 
changes from day to day and the time of day. In this way it is easier to track illegal copying 
and distribution. 

In this way simple and trusted watermarking units can be placed at trusted 
clients, which then watermark the media signal locally based on the properties that have been 
determined beforehand. A content owner can furthermore inspect the effect of the watermark 
on tiie content and then manually modify specific features p such that the watermark energy 
is weaker or stronger in different frames of the signal. In this way he can ensure that the 
watermark is not perceptible. 

The provision of the properties together with the media signal extends the 
bandwidth required for transmitting the information to the client A fourth embodiment of the 
invention is directed towards solving this problem so that no extra bandwidth is needed for 
the properties. 

A system according to this fourth embodiment of the invention is shown in a 
block schematic in Fig. 8. Fig 8 is similar to Fig. 5 and only the differences will be described 
in detail here. The server includes a lossless encoding unit LE 72 which receives the media 
signal and the properties p and losslessly codes the properties in the media signal x for 
providing a modified signal x\ The modifled signal x' is then supplied to the sending unit 52 
for sending to the client. The client device therefore also includes a lossless decoding unit 74, 
which extracts the properties p without losses from the media signal and then applies these 
properties in the watermarking unit 22. How this lossless embedding can take place is 
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described in more detail in European Patent Application No. 03100093.8, which is herein 
incorporated by reference. 

With this fourth embodiment there is no need for extra bandwidth for sending 

the properties. 

5 The present invention has many advantages apart from the ones already 

described. The watermark embedding according to the invention is particularly well suited 
for forensic tracking, where watermarks are embedded in files distributed via an Electronic 
Content Delivery System, and used to track for instance illegal copied content on the Internet. 



